Data Management Report Transformative Change Assessment Corpus - SOD

Author
Affiliation

Rainer M Krug

Published

February 19, 2024

Doi
Abstract

The literature search for the assessment corpus was conducted using search terms provided by the experts and refined in co-operation with the Knowldge and Data task force. The search was conducted using OpenAlex, scripted from R to use the API. Search terms for the following searches were defined: Transformative Change, Nature / Environment and additional search terms for individual chapters and sub-chapters To assess the quality of the corpus, sets of key-papers were selected by the experts to verify if these are in the corpus. These key-papers were selected per chapter / sub-chapter to ensure that the corpus is representative of each chapter.

Keywords

DMR, TCA, Assessment Corpus

DOI GitHub release GitHub commits since latest release License: CC BY 4.0

Working Title

IPBES_TCA_Corpus

Code repo

Github repository

Build No: 175

Introduction

The following terminology is used in this document:

  • Individual corpus: The corpus resulting from one search term, e.g. transformative or nature or ChX_Y
  • Assessment Corpus: The corpus resulting from the search terms transformative AND nature
  • Chapter corpus: The corpus resulting from transformative AND Nature AND ChX_Y

The following searches are conducted on Title and Abstrat only as the availability of fulltext drops in 2020. OpenAlex did “inherit” fro Microsoft Academic their initial corpus in 2021 which contained a lot of fulltext for searches. After that time, other sources had to be used which did not include fulltext for searches. To eliminate this bias, we linit the search for terms in abstract and title only.

Search Terms

Here are the search terms used in this document. They were provided by the authors, and some adaptations were done by the tsu to adopt them for OpenAlex.

Transformative Change

Show the code
cat(params$s_1_transformative_change)
(
    (
        (
            transformation
            OR transition *
            OR transformative
            OR "transformative change"
        )
        OR (
            (
                shift
                OR change
            )
            AND (
                fundamental
                OR deep
                OR radical
            )
        )
    )
    AND (
        socio
        OR social
        OR politics
        OR political
        OR governance
        OR economical
        OR cultural
        OR system *
        OR technological
        OR inner
        OR personal
    )
) OR (
    (
        "transformative change"
        OR "deliberate transformation*"
        OR "transformative turn*"
        OR transition *
        OR "social-ecological change*"
        OR "deep change"
        OR "fundamental alteration"
        OR "profound change"
        OR "profound transformation"
        OR "radical transformation"
        OR "transformational change"
        OR "complete change"
        OR "complete transformation"
        OR "drastic change"
        OR "in-depth transformation"
        OR "progressive change"
        OR "radical alteration"
        OR "radical change"
        OR "revolutionary change"
        OR "significant modification"
        OR "total transformation"
        OR transition
        OR pathway
        OR power
        OR agency
        OR scale
        OR leverage
        OR context
        OR process
        OR regime
        OR shift
        OR views
        OR value *
        OR structure *
        OR institution *
        OR deliberate
        OR structural
        OR fundamental
        OR system *
        OR deep
        OR radical
        OR profound
        OR drastic
        OR widespread
        OR political
        OR economical
        OR structur *
        OR complete
        OR progressive
        OR revolutionary
        OR substantial
        OR significant
    )
    AND (
        transformation
        OR alteration
        OR change
        OR turn
        OR action
        OR transition
        OR shift
    )
)

Nature

Show the code
#|

cat(params$s_1_nature_environment)
biodiversity OR marine OR terrestrial OR forest* OR woodland* OR grassland* OR savanna* OR shrubland* OR peatland OR ecosystem* OR lake* OR river* OR sea OR ocean* OR meadow* OR heathland* OR mires OR bog* OR tundra OR biosphere OR desert* OR mountain* OR "natural resource*" OR estuary OR fjord* OR fauna OR flora OR soil* OR "coastal waters" OR wetland* OR freshwater OR marshland OR marches OR dryland* OR seascape* OR landscape* OR coast* OR "arable land*" OR "agricultural land*" OR "natural environment*" OR "environmental resource*" OR agroforest* OR "agro-forest*" OR plantation* OR "protected areas" OR chaparral OR sustainable OR environment OR resilience OR conservation OR ecosystem* OR nature* OR planet* OR Earth OR biosphere OR ecological OR "socio-ecological" OR restoration OR wildlife OR landscape OR species OR bioeconomy OR "resource system" OR "coupled system" OR nature

Assessment Corpus

Show the code
#|

cat(params$s_1_f_tca_corpus())
((
    (
        (
            transformation
            OR transition *
            OR transformative
            OR "transformative change"
        )
        OR (
            (
                shift
                OR change
            )
            AND (
                fundamental
                OR deep
                OR radical
            )
        )
    )
    AND (
        socio
        OR social
        OR politics
        OR political
        OR governance
        OR economical
        OR cultural
        OR system *
        OR technological
        OR inner
        OR personal
    )
) OR (
    (
        "transformative change"
        OR "deliberate transformation*"
        OR "transformative turn*"
        OR transition *
        OR "social-ecological change*"
        OR "deep change"
        OR "fundamental alteration"
        OR "profound change"
        OR "profound transformation"
        OR "radical transformation"
        OR "transformational change"
        OR "complete change"
        OR "complete transformation"
        OR "drastic change"
        OR "in-depth transformation"
        OR "progressive change"
        OR "radical alteration"
        OR "radical change"
        OR "revolutionary change"
        OR "significant modification"
        OR "total transformation"
        OR transition
        OR pathway
        OR power
        OR agency
        OR scale
        OR leverage
        OR context
        OR process
        OR regime
        OR shift
        OR views
        OR value *
        OR structure *
        OR institution *
        OR deliberate
        OR structural
        OR fundamental
        OR system *
        OR deep
        OR radical
        OR profound
        OR drastic
        OR widespread
        OR political
        OR economical
        OR structur *
        OR complete
        OR progressive
        OR revolutionary
        OR substantial
        OR significant
    )
    AND (
        transformation
        OR alteration
        OR change
        OR turn
        OR action
        OR transition
        OR shift
    )
)) AND (biodiversity OR marine OR terrestrial OR forest* OR woodland* OR grassland* OR savanna* OR shrubland* OR peatland OR ecosystem* OR lake* OR river* OR sea OR ocean* OR meadow* OR heathland* OR mires OR bog* OR tundra OR biosphere OR desert* OR mountain* OR "natural resource*" OR estuary OR fjord* OR fauna OR flora OR soil* OR "coastal waters" OR wetland* OR freshwater OR marshland OR marches OR dryland* OR seascape* OR landscape* OR coast* OR "arable land*" OR "agricultural land*" OR "natural environment*" OR "environmental resource*" OR agroforest* OR "agro-forest*" OR plantation* OR "protected areas" OR chaparral OR sustainable OR environment OR resilience OR conservation OR ecosystem* OR nature* OR planet* OR Earth OR biosphere OR ecological OR "socio-ecological" OR restoration OR wildlife OR landscape OR species OR bioeconomy OR "resource system" OR "coupled system" OR nature)

Chapter 1

01

Show the code
#|

cat(params$s_1_ch1_01)
(
    root
    OR underlying
    OR indirect
) AND (
    driver*
    OR cause*
)

02

Show the code
#|

cat(params$s_1_ch1_02)
equity OR inequity OR just* OR unjust* OR inequality OR equality OR Fair* OR unfair*

03

Show the code
#|

cat(params$s_1_ch1_03)
scal* OR impact* OR leapfrog* OR transfer*

04

Show the code
#|

cat(params$s_1_ch1_04)
inclusi* OR particip* OR engagement* OR democratic* OR coproduct* OR transdisc* OR multiactor OR "multi-actor" OR integrat*

05

Show the code
#|

cat(params$s_1_ch1_05)
evaluat* OR reflex* OR reflect* OR monitor* OR adapt* OR learn*

06

Show the code
#|

cat(params$s_1_ch1_06)
responsib* OR accountab* OR rights OR steward* OR reciproc* OR interdependen* OR (
    relation*
    OR plural
    OR divers*
    OR "sustainability-aligned"
    OR (
        care
        AND (
            value*
            OR ethic*
        )
    )
)      

Chapter 2

Show the code
#|

cat(params$s_1_ch2)
vision OR future OR visionar* OR scenarios OR imagination OR imager* OR creativity OR desire OR wish* OR "visioning" OR "process" OR "participaory process*" OR "deliberate process*" OR "polic*" OR "target" OR "view*" OR "value" OR "cosmovision" OR "cosmocentric" OR "dream*" OR "fiction" OR "hope" OR "mission" OR "objective" OR "stORy" OR "wORldview*" OR "aspiration*" OR "action" OR "plan*" OR "strateg*" OR "intention" OR "model*" OR "solution*" OR "innovation*" OR "perspective" OR "platform" OR "collective action" OR "cooperation" OR "consultation" OR "coalition*" OR "response" OR "movement" OR "effORt" OR "initiative" OR "activity" OR "reaction" OR "perfORmance" OR "operation" OR "effect*" OR "task" OR "project" OR "influence"  OR "moment" OR "discourse" OR "motivation" OR "iteration" OR "roadmap" OR "agenda" OR "project" OR "programm" OR "government" OR "technique" OR "inspiration" OR "culture" OR "universe*" OR "reality" OR "fantasy" OR "perception" OR "visualization" OR "approach" OR "image" OR "arquetype" OR "existence" OR "cosmolog*" OR "co-production" OR "knowledge" OR "dialogue" OR "transmission" OR "conceptual*" OR "ceremon*" OR "relationships" OR "respect" OR "reciprocity" OR "responsibilities" OR "solidarity" OR "harmony" OR "self-determination" OR "communit*" OR "spiritual*" OR "languague" OR "territORy" OR "oppORtunit*" OR "sight" OR "fOResight" OR "idea" OR "appearance"

Chapter 3

01

Show the code
#|

cat(params$s_1_ch3_01)
Technolog* OR Science* OR "science-society" OR "science-technology" OR Solution

02

Show the code
#|

cat(params$s_1_ch3_02)
"co-create" OR "co-creation" OR solution* OR knowledge* OR system* OR "t-lab*" OR "technology laboratory" OR educat* OR "socio-technic*"

03

Show the code
#|

cat(params$s_1_ch3_03)
System* OR pathways OR connect* OR Agroecolog* OR Institutional OR Institutions OR Government*

04

Show the code
#|

cat(params$s_1_ch3_04)
inner OR Personal OR Religio* OR Love OR Lov* OR Feelings OR Stewardship OR Care OR Beliefs OR Belie* OR Awareness OR "Self-Awareness"

05

Show the code
#|

cat(params$s_1_ch3_05)
Worldviews OR Grassroot* OR "Community-based" OR Indigenous OR Leadership OR "Critical Science" OR Econfeminism OR "Political Ecology" OR Power OR Agency OR Environment*

06

Show the code
#|

cat(params$s_1_ch3_06)
Economi* OR "Political Economy" OR institution* OR institutions       OR govern* OR econom* OR governance OR governm* OR globalization OR states OR colonial OR coloniali* OR labour OR organization*

Chapter 4

01

Show the code
#|

cat(params$s_1_ch4_01)
(
    challenge*
    OR barrier*
    OR obstacle*
    OR hinder*
    OR hindrance*
    OR block*
    OR prevent*
    OR deter*
    OR inertia*
    OR "path dependence*"
    OR "path dependency*"
    OR stasis*
    OR "lock-in*"
    OR trap*
    OR habits*
    OR habitual*
    OR "status quo*"
    OR power*
    OR "limiting factOR*"
) AND (
    economic inequality
    OR Wealth concentration
    OR Socioeconomic inequality
    OR financialization
    OR uneven development
    OR Financialization
    OR Structural adjustment
    OR Sovereign Debt
    OR inequality
    OR Policy effectiveness
)

02

Show the code
#|

cat(params$s_1_ch4_02)
(
    challenge*
    OR barrier*
    OR obstacle*
    OR hinder*
    OR hindrance*
    OR block*
    OR prevent*
    OR deter*
    OR inertia*
    OR path dependence*
    OR path dependency*
    OR stasis*
    OR lock-in*
    OR trap*
    OR habits*
    OR habitual*
    OR status quo*
    OR power*
    OR "limiting factor*"
) AND (
    "clean technolog*"
    OR "clean innovation*"
    OR "sustainable innovation"
    OR "sustainable technological innovation*"
) AND (
    "limited access"
    OR "limited availability"
    OR "lack of access"
    OR "unavailability"
)

Chapter 5

Vision

Show the code
#|

cat(params$s_1_ch5_vision)
vision OR future OR visionar* OR scenarios OR imagination OR imager* OR creativity OR desire OR wish* OR visioning OR process OR "participatory process*" OR "deliberate process*" OR polic* OR target OR view* OR value OR cosmovision OR cosmocentric OR dream* OR fiction OR hope OR mission OR objective OR story OR worldview* OR aspiration* OR action OR plan* OR strateg* OR intention OR model* OR solution* OR innovation* OR perspective OR platform OR "collective action" OR cooperation OR consultation OR coalition* OR response OR movement OR effort OR initiative OR activity OR reaction OR performance OR operation OR effect* OR task OR project OR influence OR moment OR discourse OR motivation OR iteration OR roadmap OR agenda OR project OR programm OR government OR technique OR inspiration OR culture OR universe* OR reality OR fantasy OR perception OR visualization OR approach OR image OR arquetype OR existence OR cosmolog* OR "co-production" OR knowledge OR dialogue OR transmission OR conceptual* OR ceremon* OR relationships OR respect OR reciprocity OR responsibilities OR solidarity OR harmony OR "self-determination" OR communit* OR spiritual* OR languague OR territory OR opportunit* OR sight OR foresight OR idea OR appearance

Case

Show the code
#|

cat(params$s_1_case)
"case study*" OR case OR "study area*" OR example* OR evaluation OR concrete OR empirical OR practical OR initiative*

Vision & Case

Show the code
#|

cat(params$s_1_f_ch5_vision_case())
(vision OR future OR visionar* OR scenarios OR imagination OR imager* OR creativity OR desire OR wish* OR visioning OR process OR "participatory process*" OR "deliberate process*" OR polic* OR target OR view* OR value OR cosmovision OR cosmocentric OR dream* OR fiction OR hope OR mission OR objective OR story OR worldview* OR aspiration* OR action OR plan* OR strateg* OR intention OR model* OR solution* OR innovation* OR perspective OR platform OR "collective action" OR cooperation OR consultation OR coalition* OR response OR movement OR effort OR initiative OR activity OR reaction OR performance OR operation OR effect* OR task OR project OR influence OR moment OR discourse OR motivation OR iteration OR roadmap OR agenda OR project OR programm OR government OR technique OR inspiration OR culture OR universe* OR reality OR fantasy OR perception OR visualization OR approach OR image OR arquetype OR existence OR cosmolog* OR "co-production" OR knowledge OR dialogue OR transmission OR conceptual* OR ceremon* OR relationships OR respect OR reciprocity OR responsibilities OR solidarity OR harmony OR "self-determination" OR communit* OR spiritual* OR languague OR territory OR opportunit* OR sight OR foresight OR idea OR appearance) AND ("case study*" OR case OR "study area*" OR example* OR evaluation OR concrete OR empirical OR practical OR initiative*)

Topics

OpenAlex assigns topics to each work in a hirarchical manner:

Please see here for more information and here for a complete list of all topics and their corresponding subfields, fields and domains.

Get and calculate Data

In this section, the data is retrieved from OpenAlex and the calculations are done. It contains the code used. No results are shown here, so this section can be skipped.

Show the code
#|

fn <- file.path(".", "data", "search_term_hits.rds")
if (!file.exists(fn)) {
    s_t <- grep("s_1_", names(params), value = TRUE)
    search_term_hits <- parallel::mclapply(
        s_t,
        function(stn) {
            message("getting '", stn, "' ...")
            if (grepl("_f_", stn)) {
                search <- params[[stn]]()
            } else {
                search <- params[[stn]]
            }
            search <- compact(search)
            openalexR::oa_query(filter = list(title_and_abstract.search = search)) |>
                openalexR::oa_request(count_only = TRUE, verbose = TRUE) |>
                unlist()
        },
        mc.cores = params$mc.cores,
        mc.preschedule = FALSE
    ) |>
        do.call(what = cbind) |>
        t() |>
        as.data.frame() |>
        dplyr::mutate(page = NULL, per_page = NULL) |>
        dplyr::mutate(count = formatC(count, format = "f", big.mark = ",", digits = 0))

    rownames(search_term_hits) <- s_t |>
        gsub(pattern = "s_1_", replacement = "") |>
        gsub(pattern = "f_", replacement = "") |>
        gsub(pattern = "^ch", replacement = "Chapter ") |>
        gsub(pattern = "_", replacement = " ")

    saveRDS(search_term_hits, file = fn)
} else {
    search_term_hits <- readRDS(fn)
}
Show the code
#|

fn <- file.path(".", "data", "countries_tca_corpus.rds")
if (!file.exists(fn)) {
    countries_tca_corpus <- openalexR::oa_query(
        filter = list(
            title_and_abstract.search = compact(params$s_1_f_tca_corpus())
        ),
        group_by = "authorships.countries"
    ) |>
        openalexR::oa_request(count_only = FALSE, verbose = TRUE) |>
        sapply(FUN = unlist) |>
        t() |>
        as.data.frame() |>
        dplyr::rename(iso2c = key) |>
        mutate(count = as.numeric(count))

    saveRDS(countries_tca_corpus, file = fn)
} else {
    countries_tca_corpus <- readRDS(fn)
}
Show the code
fn <- file.path(".", "data", paste0("sample_tca_corpus_", params$sample_size, ".rds"))
fn_df <- file.path(".", "data", paste0("sample_tca_corpus_", params$sample_size, ".rds"))
if (!file.exists(fn)) {
    message("Sampling 'transformative change' corpus (n = ", params$sample_size, ") - this can take some time ...")
    sample_tca_corpus <- openalexR::oa_query(
        filter = list(
            title_and_abstract.search = compact(
            paste0(
                "(", params$s_1_transformative_change, ") AND (", params$s_1_nature_environment, ")"
            )
            )
        ),
        options = list(
            sample = params$sample_size
        )
    ) |>
        openalexR::oa_request(count_only = FALSE, verbose = TRUE)

    sample_tca_corpus_df <- oa2df(sample_tca_corpus, entity = "works")

    saveRDS(sample_tca_corpus, file = fn)
    saveRDS(sample_tca_corpus_df, file = fn_df)
} else {
    sample_tca_corpus <- readRDS(fn)
    sample_tca_corpus_df <- readRDS(fn_df)
}
Show the code
fn <- file.path(".", "data", paste0("prim_topics_tca_corpus.rds"))
if (!file.exists(fn)) {
    # message("Sampling 'transformative change' corpus (n = ", params$sample_size, ") - this can take some time ...")

    prim_topics_tca_corpus <- openalexR::oa_fetch(
        listsearch = compact(paste0("(", params$s_1_transformative_change, ") AND (", params$s_1_nature_environment, ")")),
        group_by = "primary_topic.id",
        verbose = FALSE
    )
    names(prim_topics_tca_corpus) <- c("topic_id", "topic_name", "count")
    prim_topics_tca_corpus <- prim_topics_tca_corpus[-2]
    prim_topics_tca_corpus$topic_id <- sub("https://openalex.org/T", "", prim_topics_tca_corpus$topic_id)
    prim_topics_tca_corpus <- merge(
        read.csv(file.path("inputs", "OpenAlex_topic_mapping_table - final_topic_field_subfield_table.csv")),
        prim_topics_tca_corpus,
        by = "topic_id"
    )

    prim_topics_tca_corpus <- prim_topics_tca_corpus[order(prim_topics_tca_corpus$count, decreasing = TRUE), ]

    saveRDS(prim_topics_tca_corpus, file = fn)
} else {
    prim_topics_tca_corpus <- readRDS(fn)
}
Show the code
#|

fn <- file.path(".", "data", paste0("nature_corpus_plus.rds"))
if (!file.exists(fn)) {
    nature_corpus_plus <- openalexR::oa_fetch(
        title_and_abstract.search = compact(params$s_1_nature_environment),
        concepts.id = paste(
            "C86803240",
            "C18903297",
            "C205649164",
            "C59822182",
            "C127313418",
            "C151730666",
            "C90856448",
            "C159390177",
            "C78458016",
            "C58640448",
            "C185933670",
            "C62649853",
            "C114793014",
            "C97137747",
            "C132651083",
            "C110872660",
            "C24518262",
            "C130217890",
            "C43827410",
            sep = "|"
        ),
        sustainable_development_goals.id = paste(
            "https://metadata.un.org/sdg/14",
            "https://metadata.un.org/sdg/15",
            "https://metadata.un.org/sdg/13",
            sep = "|"
        ),
        options = list(
            select = "id, authorships"
        ),
        verbose = TRUE,
        count_only = TRUE
    )
    save(nature_corpus_plus, file = fn)
} else {
    nature_corpus_plus <- readRDS(fn)
}
Show the code
#|

fn <- file.path(".", "data", paste0("tc_corpus_plus.rds"))
if (!file.exists(fn)) {
    tc_corpus_plus <- openalexR::oa_fetch(
        title_and_abstract.search = compact(params$s_1_transformative_change),
        concepts.id = paste(
            "C548081761",
            "C162324750",
            "C87717796",
            "C139719470",
            "C2778348673",
            "C175605778",
            "C47737302",
            "C17744445",
            "C199539241",
            "C144133560",
            "C138885662",
            "C94625758",
            "C10138342",
            "C50522688",
            "C162853370",
            "C77805123",
            "C36289849",
            "C39549134",
            "C187736073",
            "C188147891",
            "C66204764",
            "C199033989",
            "C74363100",
            "C2776867660",
            "C134560507",
            "C106159729",
            "C4249254",
            "C47768531",
            "C190248442",
            "C95124753",
            "C6303427",
            sep = "|"
        ),
        sustainable_development_goals.id = paste(
            "https://metadata.un.org/sdg/16",
            "https://metadata.un.org/sdg/10",
            "https://metadata.un.org/sdg/5",
            "https://metadata.un.org/sdg/4",
            "https://metadata.un.org/sdg/8",
            "https://metadata.un.org/sdg/1",
            "https://metadata.un.org/sdg/3",
            "https://metadata.un.org/sdg/2",
            "https://metadata.un.org/sdg/12",
            "https://metadata.un.org/sdg/9",
            "https://metadata.un.org/sdg/17",
            "https://metadata.un.org/sdg/11",
            "https://metadata.un.org/sdg/6",
            "https://metadata.un.org/sdg/7",
            sep = "|"
        ),
        options = list(
            select = "id, authorships"
        ),
        verbose = TRUE,
        count_only = TRUE
    )
    save(tc_corpus_plus, file = fn)
} else {
    tc_corpus_plus <- readRDS(fn)
}
Show the code
#|
fn <- file.path(".", "data", "key_papers.rds")
if (!file.exists(fn)) {
    key_papers <- lapply(
        params$key_papers,
        function(fn) {
            message("Processing '", fn, "' ...")
            sapply(
                fn,
                function(x) {
                    read.csv(x) |>
                        select(DOI)
                }
            ) |>
                unlist()
        }
    )
    names(key_papers) <- gsub("\\.csv", "", basename(params$key_papers))

    key_papers <- list(
        Ch_1 = unlist(key_papers[grepl("Ch 1 -", names(key_papers))], recursive = FALSE) |> as.vector(),
        Ch_2 = unlist(key_papers[grepl("Ch 2 -", names(key_papers))], recursive = FALSE) |> as.vector(),
        Ch_3_Cl_1 = unlist(key_papers[grepl("Ch 3 - Cl1", names(key_papers))], recursive = FALSE) |> as.vector(),
        Ch_3_Cl_3 = unlist(key_papers[grepl("Ch 3 - Cl3", names(key_papers))], recursive = FALSE) |> as.vector(),
        Ch_3_Cl_4 = unlist(key_papers[grepl("Ch 3 - Cl4", names(key_papers))], recursive = FALSE) |> as.vector(),
        Ch_3_Cl_5 = unlist(key_papers[grepl("Ch 3 - Cl5", names(key_papers))], recursive = FALSE) |> as.vector(),
        Ch_3_Cl_6 = unlist(key_papers[grepl("Ch 3 - Cl6", names(key_papers))], recursive = FALSE) |> as.vector(),
        Ch_3 = unlist(key_papers[grepl("Ch 3 - p", names(key_papers))], recursive = FALSE) |> as.vector(),
        Ch_4_Cl_1 = unlist(key_papers[grepl("Ch 4 - Challenge 1", names(key_papers))], recursive = FALSE) |> as.vector(),
        Ch_4_Cl_2 = unlist(key_papers[grepl("Ch 4 - Challenge 2", names(key_papers))], recursive = FALSE) |> as.vector(),
        Ch_4_Cl_3 = unlist(key_papers[grepl("Ch 4 - Challenge 3", names(key_papers))], recursive = FALSE) |> as.vector(),
        Ch_4_Cl_4 = unlist(key_papers[grepl("Ch 4 - Challenge 4", names(key_papers))], recursive = FALSE) |> as.vector(),
        Ch_4_Cl_5 = unlist(key_papers[grepl("Ch 4 - Challenge 5", names(key_papers))], recursive = FALSE) |> as.vector(),
        Ch_5 = unlist(key_papers[grepl("Ch 5 -", names(key_papers))], recursive = FALSE) |> as.vector()
    )

    saveRDS(key_papers, file = fn)
} else {
    key_papers <- readRDS(fn)
}
Show the code
#|

fn_kw <- file.path(".", "data", "key_works.rds")
fn_kw_df <- file.path(".", "data", "key_works_df.rds")
if (!all(file.exists(fn_kw, fn_kw_df))) {
    key_works <- parallel::mclapply(
        key_papers,
        function(kp) {
            dois <- kp[kp != ""] |>
                unlist() |>
                tolower() |>
                unique()

            openalexR::oa_fetch(doi = dois, output = "list")
        },
        mc.cores = params$mc.cores,
        mc.preschedule = FALSE
    )

    found <- sapply(
        key_works,
        function(x) {
            length(x) > 0
        }
    )

    key_works <- key_works[found]

    print("The following key paper sets were excluded as they contained no papers in OpenAlex:\n")
    print(names(found)[!found])

    saveRDS(key_works, file = fn_kw)

    key_works_df <- lapply(
        key_works,
        oa2df,
        entity = "works"
    )

    saveRDS(key_works_df, fn_kw_df)
} else {
    key_works <- readRDS(file = fn_kw)
    key_works_df <- readRDS(fn_kw_df)
}
Show the code
#|

fn <- file.path(".", "data", "key_works_hits.rds")
if (!file.exists(fn)) {
    kws <- key_works_df
    kws$all  <- key_works_df |>
        bind_rows()

    nms <- names(kws)
    key_works_hits <- pbapply::pblapply(
        nms,
        function(nm) {
            message("Getting key paper set for ", nm, " ...")
            dois <- kws[[nm]] |>
                select(doi) |>
                distinct() |>
                unlist() |>
                unique() |>
                tolower()

            s_t <- grep("s_1_", names(params), value = TRUE)
            kw_h <- parallel::mclapply(
                s_t,
                function(stn) {
                    message("  getting '", stn, "' ...")
                    if (grepl("_f_", stn)) {
                        search <- compact(params[[stn]]())
                    } else {
                        search <- compact(params[[stn]])
                    }
                    get_count(dois = dois, list(title_and_abstract.search = search), verbose = FALSE)
                },
                mc.cores = params$mc.cores,
                mc.preschedule = FALSE
            ) |>
                do.call(what = cbind) |>
                as.data.frame()

            names(kw_h) <- s_t

            # if (ncol(kw_h) == 1){
            #     kw_h <- t(kw_h)
            #     rownames(kw_h) <- dois
            # }

            kw_h <- rbind(
                kw_h,
                colSums(kw_h)
            )

            rownames(kw_h)[[nrow(kw_h)]] <- "Total"
            return(kw_h)  
        }
    )

    names(key_works_hits) <- nms

    for (i in nms) {
        # key_works_hits[[i]] <- cbind(
        #     key_works_hits[[i]],
        #     key_works_hits_tca_filtered[[i]]
        # )

        key_works_hits[[i]] <- cbind(
            key_works_hits[[i]],
            Total = rowSums(key_works_hits[[i]])
        ) |>
            mutate(Total = Total - 1) # |>
            # relocate(tca_corpus_SDG, .after = s_1_f_tca_corpus)
    }

    ###

    saveRDS(key_works_hits, file = fn)
} else {
    key_works_hits <- readRDS(file = fn)
}

Results

Number of Hits per Individual Corpus

Here we show the number of hits for the key papers in the different individual corpi. The columns represent the different search terms as defined in Section 2.1.

Show the code
dat <- cbind(
    search_term_hits
)

rownames(dat) <- dplyr::recode(
    rownames(dat),
    "transformative change" = "Transformative Change @sec-transform",
    "nature environment" = "Nature @sec-nature",
    "tca corpus" = "Assessment Corpus @sec-tca-corpus",
    "Chapter 1 01" = "Ch1 01 @sec-ch1-01",
    "Chapter 1 02" = "Ch1 02 @sec-ch1-02",
    "Chapter 1 03" = "Ch1 03 @sec-ch1-03",
    "Chapter 1 04" = "Ch1 04 @sec-ch1-04",
    "Chapter 1 05" = "Ch1 05 @sec-ch1-05",
    "Chapter 1 06" = "Ch1 06 @sec-ch1-06",
    "Chapter 2" = "Ch2  @sec-ch2",
    "Chapter 3 01" = "Ch3 01 @sec-ch3-01",
    "Chapter 3 02" = "Ch3 02 @sec-ch3-02",
    "Chapter 3 03" = "Ch3 03 @sec-ch3-03",
    "Chapter 3 04" = "Ch3 04 @sec-ch3-04",
    "Chapter 3 05" = "Ch3 05 @sec-ch3-05",
    "Chapter 3 06" = "Ch3 06 @sec-ch3-06",
    "Chapter 4 01" = "Ch4 01 @sec-ch4-01",
    "Chapter 4 02" = "Ch4 02 @sec-ch4-02",
    "Chapter 5 vision" = "Ch5 Vision @sec-ch5_vision",
    "Chapter 5 vision case" = "Ch5 Vision Case @sec-ch5_vision_case",
    "case" = "Ch5 Case @sec-case"
)

dat |>
    knitr::kable(
        caption = "Number of hits",
    )
Number of hits
count db_response_time_ms
oa 248,825,624 50
Transformative Change Section 2.1.1 18,226,502 5366
Nature Section 2.1.2 24,259,266 3729
Assessment Corpus Section 2.1.3 4,555,277 6131
Ch1 01 Section 2.1.4.1 620,287 263
Ch1 02 Section 2.1.4.2 3,160,978 372
Ch1 03 Section 2.1.4.3 10,616,048 507
Ch1 04 Section 2.1.4.4 1,035,650 451
Ch1 05 Section 2.1.4.5 13,182,471 567
Ch1 06 Section 2.1.4.6 4,479,437 842
Ch2 Section 2.1.5 102,935,605 8371
Ch3 01 Section 2.1.6.1 10,484,517 1063
Ch3 02 Section 2.1.6.2 29,626,219 2153
Ch3 03 Section 2.1.6.3 28,300,860 1419
Ch3 04 Section 2.1.6.4 9,140,653 1620
Ch3 05 Section 2.1.6.5 12,933,146 1300
Ch3 06 Section 2.1.6.6 15,103,257 1168
Ch4 01 Section 2.1.7.1 45,078 1034
Ch4 02 Section 2.1.7.2 9 750
Ch5 Vision Section 2.1.8.1 102,935,618 6554
Ch5 Case Section 2.1.8.2 34,184,663 2609
Ch5 Vision Case Section 2.1.8.3 26,484,781 5009
Show the code
rm(dat)

Key papers in different Individual Corpi

Show the code
#|

tbl <- lapply(
    names(key_works_hits),
    function(n) {
        kwh <- key_works_hits[[n]]
        if (nrow(kwh) > 0) {
            total <- grepl("Total", rownames(kwh))
            rownames(kwh)[!total] <- paste0(n, " - <a href='https://doi.org/", rownames(kwh)[!total], "' target='_blank'>Click here</a>")
            rownames(kwh)[total] <- paste0("**", n, " - Total**")
            kwh |>
                arrange(Total) |>
                apply(
                    c(1, 2),
                    function(x) {
                        ifelse(x == 0, "<font color='red'>0</font>", paste0("<font color='green'>", x, "</font>"))
                    }
                ) |>
                as.data.frame()
        } else {
            return(NULL)
        }
    }
)
tbl <- tbl[sapply(tbl, class) != "NULL"]
tbl <- do.call(what = rbind, tbl)


detail <- rbind(
    "**overall**" = c(
        paste0(
            "**",
            search_term_hits |>
                select(count) |>
                unlist() |>
                as.vector(),
            "**"
        ),
        ""
    ),
    tbl
)

detail <- detail |>
    dplyr::rename(
        "Transformative Change @sec-transform" = s_1_transformative_change,
        "Nature @sec-nature" = s_1_nature_environment,
        "Assessment Corpus @sec-tca-corpus" = s_1_f_tca_corpus,
        "Ch1 01 @sec-ch1-01" = s_1_ch1_01,
        "Ch1 02 @sec-ch1-02" = s_1_ch1_02,
        "Ch1 03 @sec-ch1-03" = s_1_ch1_03,
        "Ch1 04 @sec-ch1-04" = s_1_ch1_04,
        "Ch1 05 @sec-ch1-05" = s_1_ch1_05,
        "Ch1 06 @sec-ch1-06" = s_1_ch1_06,
        "Ch2  @sec-ch2" = s_1_ch2,
        "Ch3 01 @sec-ch3-01" = s_1_ch3_01,
        "Ch3 02 @sec-ch3-02" = s_1_ch3_02,
        "Ch3 03 @sec-ch3-03" = s_1_ch3_03,
        "Ch3 04 @sec-ch3-04" = s_1_ch3_04,
        "Ch3 05 @sec-ch3-05" = s_1_ch3_05,
        "Ch3 06 @sec-ch3-06" = s_1_ch3_06,
        "Ch4 01 @sec-ch4-01" = s_1_ch4_01,
        "Ch4 02 @sec-ch4-02" = s_1_ch4_02,
        "Ch5 Vision @sec-ch5_vision" = s_1_ch5_vision,
        "Ch5 Case @sec-case" = s_1_case,
        "Ch5 Vision Case @sec-ch5_vision_case" = s_1_f_ch5_vision_case
    )

Key Papers in Individual Corpi

Summary

Each column is a different search term, and each row consists of the key papers of a specific chapter and the author who provided the key papers. The number is the number of key papers occurring in the Individual Corpus.

Show the code
in_summary <- grepl("Total|overall", rownames(detail))
knitr::kable(
    detail[in_summary, ]
)
s_1_oa Transformative Change Section 2.1.1 Nature Section 2.1.2 Assessment Corpus Section 2.1.3 Ch1 01 Section 2.1.4.1 Ch1 02 Section 2.1.4.2 Ch1 03 Section 2.1.4.3 Ch1 04 Section 2.1.4.4 Ch1 05 Section 2.1.4.5 Ch1 06 Section 2.1.4.6 Ch2 Section 2.1.5 Ch3 01 Section 2.1.6.1 Ch3 02 Section 2.1.6.2 Ch3 03 Section 2.1.6.3 Ch3 04 Section 2.1.6.4 Ch3 05 Section 2.1.6.5 Ch3 06 Section 2.1.6.6 Ch4 01 Section 2.1.7.1 Ch4 02 Section 2.1.7.2 Ch5 Vision Section 2.1.8.1 Ch5 Case Section 2.1.8.2 Ch5 Vision Case Section 2.1.8.3 Total
overall 248,825,624 18,226,502 24,259,266 4,555,277 620,287 3,160,978 10,616,048 1,035,650 13,182,471 4,479,437 102,935,605 10,484,517 29,626,219 28,300,860 9,140,653 12,933,146 15,103,257 45,078 9 102,935,618 34,184,663 26,484,781
Ch_1 - Total 42 37 39 35 10 18 15 14 17 10 40 21 23 24 11 23 24 4 0 40 25 25 496
Ch_2 - Total 22 17 22 17 3 12 8 3 7 7 21 11 13 14 10 13 18 0 0 21 11 11 260
Ch_3_Cl_3 - Total 4 4 4 4 3 1 3 1 4 4 4 4 4 4 4 4 4 0 0 4 4 4 71
Ch_3_Cl_4 - Total 5 5 4 4 2 3 2 1 2 3 4 4 5 5 4 3 3 1 0 4 3 3 69
Ch_3_Cl_5 - Total 3 2 3 2 0 1 0 0 1 0 2 2 0 1 0 1 1 0 0 2 1 1 22
Ch_3_Cl_6 - Total 6 6 5 5 1 1 1 3 1 2 5 2 4 5 1 1 5 0 0 5 3 3 64
Ch_3 - Total 4 4 4 4 2 1 2 2 3 3 4 3 4 3 1 2 2 0 0 4 4 4 59
Ch_4_Cl_1 - Total 7 2 7 2 0 2 3 2 3 4 7 3 3 3 2 6 6 0 0 7 3 3 74
Ch_4_Cl_2 - Total 4 2 3 2 1 2 1 0 1 0 3 1 2 2 1 2 3 0 0 3 2 2 36
Ch_4_Cl_3 - Total 5 5 5 5 0 1 1 0 1 2 4 1 3 3 1 3 4 0 0 4 4 4 55
Ch_4_Cl_4 - Total 4 2 4 2 1 1 3 1 1 1 4 2 1 2 2 2 4 0 0 4 3 3 46
Ch_4_Cl_5 - Total 5 3 4 3 1 1 2 1 1 0 5 2 3 3 1 2 4 1 0 5 4 4 54
Ch_5 - Total 35 31 32 28 10 15 17 9 17 11 35 22 28 29 20 24 20 2 0 35 19 19 457
all - Total 134 109 125 103 32 54 54 35 56 43 127 71 84 89 52 80 92 6 0 127 79 79 1630

Detail

Show the code
knitr::kable(
    detail
)
s_1_oa Transformative Change Section 2.1.1 Nature Section 2.1.2 Assessment Corpus Section 2.1.3 Ch1 01 Section 2.1.4.1 Ch1 02 Section 2.1.4.2 Ch1 03 Section 2.1.4.3 Ch1 04 Section 2.1.4.4 Ch1 05 Section 2.1.4.5 Ch1 06 Section 2.1.4.6 Ch2 Section 2.1.5 Ch3 01 Section 2.1.6.1 Ch3 02 Section 2.1.6.2 Ch3 03 Section 2.1.6.3 Ch3 04 Section 2.1.6.4 Ch3 05 Section 2.1.6.5 Ch3 06 Section 2.1.6.6 Ch4 01 Section 2.1.7.1 Ch4 02 Section 2.1.7.2 Ch5 Vision Section 2.1.8.1 Ch5 Case Section 2.1.8.2 Ch5 Vision Case Section 2.1.8.3 Total
overall 248,825,624 18,226,502 24,259,266 4,555,277 620,287 3,160,978 10,616,048 1,035,650 13,182,471 4,479,437 102,935,605 10,484,517 29,626,219 28,300,860 9,140,653 12,933,146 15,103,257 45,078 9 102,935,618 34,184,663 26,484,781
Ch_1 - Click here 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Ch_1 - Click here 1 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 3
Ch_1 - Click here 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 3
Ch_1 - Click here 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 3
Ch_1 - Click here 1 0 1 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 1 0 0 4
Ch_1 - Click here 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 5
Ch_1 - Click here 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 5
Ch_1 - Click here 1 1 1 1 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 1 0 0 6
Ch_1 - Click here 1 1 1 1 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 6
Ch_1 - Click here 1 1 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 1 1 1 6
Ch_1 - Click here 1 1 1 1 0 0 0 1 0 0 1 0 0 0 0 1 0 0 0 1 0 0 7
Ch_1 - Click here 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0 1 1 0 0 1 0 0 7
Ch_1 - Click here 1 1 1 1 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 1 0 0 7
Ch_1 - Click here 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 1 1 7
Ch_1 - Click here 1 1 1 1 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 1 1 1 8
Ch_1 - Click here 1 1 0 0 0 1 0 0 0 0 1 1 1 1 1 0 1 0 0 1 0 0 9
Ch_1 - Click here 1 1 1 1 0 0 0 0 0 1 1 0 1 1 0 0 0 0 0 1 1 1 10
Ch_1 - Click here 1 1 1 1 0 1 0 0 1 0 1 1 1 1 0 0 0 0 0 1 0 0 10
Ch_1 - Click here 1 1 1 1 0 0 0 0 1 0 1 1 1 0 0 0 0 0 0 1 1 1 10
Ch_1 - Click here 1 1 1 1 0 1 0 0 0 1 1 0 0 1 0 1 1 0 0 1 0 0 10
Ch_1 - Click here 1 1 1 1 1 0 0 0 1 0 1 0 0 0 0 0 1 0 0 1 1 1 10
Ch_1 - Click here 1 1 1 1 0 0 0 0 1 0 1 1 1 0 0 0 0 0 0 1 1 1 10
Ch_1 - Click here 1 1 1 1 0 0 1 1 0 0 1 0 0 1 0 1 1 0 0 1 0 0 10
Ch_1 - Click here 1 1 1 1 0 0 0 0 0 0 1 0 1 1 0 1 1 0 0 1 1 1 11
Ch_1 - Click here 1 1 1 1 0 0 0 0 1 0 1 0 0 1 1 0 1 0 0 1 1 1 11
Ch_1 - Click here 1 1 1 1 0 0 1 0 0 0 1 1 0 1 0 1 0 0 0 1 1 1 11
Ch_1 - Click here 1 1 1 1 0 1 1 0 0 0 1 1 1 1 0 1 1 0 0 1 0 0 12
Ch_1 - Click here 1 1 1 1 0 0 0 1 0 1 1 1 1 1 0 1 1 0 0 1 1 1 14
Ch_1 - Click here 1 1 1 1 0 1 0 0 0 0 1 1 1 1 1 1 1 0 0 1 1 1 14
Ch_1 - Click here 1 1 1 1 0 1 0 1 1 0 1 1 1 1 0 1 1 0 0 1 1 1 15
Ch_1 - Click here 1 1 1 1 0 1 0 1 1 0 1 1 1 1 0 1 1 0 0 1 1 1 15
Ch_1 - Click here 1 1 1 1 1 1 1 0 1 0 1 1 1 1 0 1 1 0 0 1 1 1 16
Ch_1 - Click here 1 1 1 1 1 1 1 0 1 0 1 0 1 1 0 1 1 1 0 1 1 1 16
Ch_1 - Click here 1 1 1 1 1 0 0 1 1 1 1 1 1 1 0 1 1 0 0 1 1 1 16
Ch_1 - Click here 1 1 1 1 0 1 1 1 0 0 1 1 1 1 1 1 1 0 0 1 1 1 16
Ch_1 - Click here 1 1 1 1 0 0 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 17
Ch_1 - Click here 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 18
Ch_1 - Click here 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 19
Ch_1 - Click here 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 19
Ch_1 - Click here 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 19
Ch_1 - Click here 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 20
Ch_1 - Click here 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 20
Ch_1 - Total 42 37 39 35 10 18 15 14 17 10 40 21 23 24 11 23 24 4 0 40 25 25 496
Ch_2 - Click here 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 3
Ch_2 - Click here 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 1 0 0 4
Ch_2 - Click here 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 5
Ch_2 - Click here 1 0 1 0 0 1 1 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 5
Ch_2 - Click here 1 1 1 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 1 0 0 6
Ch_2 - Click here 1 0 1 0 0 0 0 0 0 1 1 0 0 0 1 0 1 0 0 1 0 0 6
Ch_2 - Click here 1 0 1 0 0 0 0 0 0 0 1 1 0 0 0 1 1 0 0 1 1 1 8
Ch_2 - Click here 1 1 1 1 0 1 0 0 0 0 1 0 1 1 0 0 1 0 0 1 0 0 9
Ch_2 - Click here 1 1 1 1 0 0 0 0 0 0 1 1 0 0 0 0 1 0 0 1 1 1 9
Ch_2 - Click here 1 1 1 1 0 0 0 0 0 0 1 1 1 1 0 1 0 0 0 1 0 0 9
Ch_2 - Click here 1 1 1 1 0 1 0 0 0 0 1 0 1 1 1 0 1 0 0 1 0 0 10
Ch_2 - Click here 1 1 1 1 0 0 0 0 0 0 1 0 1 1 0 1 1 0 0 1 1 1 11
Ch_2 - Click here 1 1 1 1 0 1 0 0 0 1 1 0 0 0 0 1 1 0 0 1 1 1 11
Ch_2 - Click here 1 1 1 1 0 1 0 0 1 0 1 1 1 1 1 1 1 0 0 1 0 0 13
Ch_2 - Click here 1 1 1 1 0 1 0 0 0 1 1 1 1 1 1 1 1 0 0 1 0 0 13
Ch_2 - Click here 1 1 1 1 0 1 1 0 1 0 1 1 1 1 0 1 1 0 0 1 1 1 15
Ch_2 - Click here 1 1 1 1 0 0 1 0 1 1 1 1 1 1 1 1 1 0 0 1 1 1 16
Ch_2 - Click here 1 1 1 1 0 1 1 0 1 0 1 1 1 1 1 1 1 0 0 1 1 1 16
Ch_2 - Click here 1 1 1 1 0 1 1 1 1 0 1 1 1 1 1 1 1 0 0 1 1 1 17
Ch_2 - Click here 1 1 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 0 0 1 1 1 17
Ch_2 - Click here 1 1 1 1 1 1 1 1 1 1 1 0 1 1 0 1 1 0 0 1 1 1 17
Ch_2 - Click here 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 19
Ch_2 - Total 22 17 22 17 3 12 8 3 7 7 21 11 13 14 10 13 18 0 0 21 11 11 260
Ch_3_Cl_3 - Click here 1 1 1 1 0 0 1 0 1 1 1 1 1 1 1 1 1 0 0 1 1 1 16
Ch_3_Cl_3 - Click here 1 1 1 1 1 0 1 0 1 1 1 1 1 1 1 1 1 0 0 1 1 1 17
Ch_3_Cl_3 - Click here 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 17
Ch_3_Cl_3 - Click here 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 0 0 1 1 1 18
Ch_3_Cl_3 - Total 4 4 4 4 3 1 3 1 4 4 4 4 4 4 4 4 4 0 0 4 4 4 71
Ch_3_Cl_4 - Click here 1 1 1 1 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 6
Ch_3_Cl_4 - Click here 1 1 0 0 0 1 0 0 0 0 1 1 1 1 1 0 1 0 0 1 0 0 9
Ch_3_Cl_4 - Click here 1 1 1 1 0 0 0 0 0 1 1 0 1 1 1 1 0 0 0 1 1 1 12
Ch_3_Cl_4 - Click here 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 0 0 1 1 1 18
Ch_3_Cl_4 - Click here 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 20
Ch_3_Cl_4 - Total 5 5 4 4 2 3 2 1 2 3 4 4 5 5 4 3 3 1 0 4 3 3 69
Ch_3_Cl_5 - Click here 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
Ch_3_Cl_5 - Click here 1 1 1 1 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 1 0 0 6
Ch_3_Cl_5 - Click here 1 1 1 1 0 1 0 0 1 0 1 1 0 1 0 1 1 0 0 1 1 1 13
Ch_3_Cl_5 - Total 3 2 3 2 0 1 0 0 1 0 2 2 0 1 0 1 1 0 0 2 1 1 22
Ch_3_Cl_6 - Click here 1 1 1 1 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 5
Ch_3_Cl_6 - Click here 1 1 0 0 0 0 0 0 0 0 1 1 1 1 0 0 1 0 0 1 0 0 7
Ch_3_Cl_6 - Click here 1 1 1 1 0 0 0 1 0 0 1 0 0 1 0 0 1 0 0 1 0 0 8
Ch_3_Cl_6 - Click here 1 1 1 1 0 0 0 0 0 0 1 0 1 1 0 0 1 0 0 1 1 1 10
Ch_3_Cl_6 - Click here 1 1 1 1 0 0 0 1 0 1 1 0 0 0 0 0 1 0 0 1 1 1 10
Ch_3_Cl_6 - Click here 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 19
Ch_3_Cl_6 - Total 6 6 5 5 1 1 1 3 1 2 5 2 4 5 1 1 5 0 0 5 3 3 64
Ch_3 - Click here 1 1 1 1 0 0 0 0 0 1 1 0 1 1 0 0 0 0 0 1 1 1 10
Ch_3 - Click here 1 1 1 1 0 0 0 0 1 0 1 1 1 0 0 0 0 0 0 1 1 1 10
Ch_3 - Click here 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 18
Ch_3 - Click here 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 0 0 1 1 1 18
Ch_3 - Total 4 4 4 4 2 1 2 2 3 3 4 3 4 3 1 2 2 0 0 4 4 4 59
Ch_4_Cl_1 - Click here 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 1 0 0 4
Ch_4_Cl_1 - Click here 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 1 1 0 0 1 0 0 5
Ch_4_Cl_1 - Click here 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 1 1 1 6
Ch_4_Cl_1 - Click here 1 0 1 0 0 0 1 0 0 1 1 0 0 0 0 1 1 0 0 1 0 0 7
Ch_4_Cl_1 - Click here 1 0 1 0 0 0 0 0 1 1 1 1 1 1 0 1 1 0 0 1 0 0 10
Ch_4_Cl_1 - Click here 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 18
Ch_4_Cl_1 - Click here 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 18
Ch_4_Cl_1 - Total 7 2 7 2 0 2 3 2 3 4 7 3 3 3 2 6 6 0 0 7 3 3 74
Ch_4_Cl_2 - Click here 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Ch_4_Cl_2 - Click here 1 0 1 0 0 1 0 0 0 0 1 0 0 0 0 0 1 0 0 1 1 1 7
Ch_4_Cl_2 - Click here 1 1 1 1 0 0 0 0 0 0 1 0 1 1 0 1 1 0 0 1 0 0 9
Ch_4_Cl_2 - Click here 1 1 1 1 1 1 1 0 1 0 1 1 1 1 1 1 1 0 0 1 1 1 17
Ch_4_Cl_2 - Total 4 2 3 2 1 2 1 0 1 0 3 1 2 2 1 2 3 0 0 3 2 2 36
Ch_4_Cl_3 - Click here 1 1 1 1 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 5
Ch_4_Cl_3 - Click here 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 1 1 1 8
Ch_4_Cl_3 - Click here 1 1 1 1 0 0 0 0 0 1 1 0 0 0 0 1 1 0 0 1 1 1 10
Ch_4_Cl_3 - Click here 1 1 1 1 0 0 0 0 0 0 1 0 1 1 0 1 1 0 0 1 1 1 11
Ch_4_Cl_3 - Click here 1 1 1 1 0 1 1 0 1 1 1 1 1 1 1 1 1 0 0 1 1 1 17
Ch_4_Cl_3 - Total 5 5 5 5 0 1 1 0 1 2 4 1 3 3 1 3 4 0 0 4 4 4 55
Ch_4_Cl_4 - Click here 1 0 1 0 0 0 1 1 0 0 1 0 0 0 0 0 1 0 0 1 0 0 6
Ch_4_Cl_4 - Click here 1 1 1 1 0 1 0 0 0 0 1 1 0 0 0 0 1 0 0 1 1 1 10
Ch_4_Cl_4 - Click here 1 0 1 0 0 0 1 0 0 0 1 0 0 1 1 1 1 0 0 1 1 1 10
Ch_4_Cl_4 - Click here 1 1 1 1 1 0 1 0 1 1 1 1 1 1 1 1 1 0 0 1 1 1 17
Ch_4_Cl_4 - Total 4 2 4 2 1 1 3 1 1 1 4 2 1 2 2 2 4 0 0 4 3 3 46
Ch_4_Cl_5 - Click here 1 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 1 0 0 1 1 1 6
Ch_4_Cl_5 - Click here 1 0 1 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 1 0 0 6
Ch_4_Cl_5 - Click here 1 1 1 1 0 0 1 0 0 0 1 0 0 0 0 1 1 0 0 1 1 1 10
Ch_4_Cl_5 - Click here 1 1 1 1 0 1 0 0 0 0 1 0 1 1 0 0 1 0 0 1 1 1 11
Ch_4_Cl_5 - Click here 1 1 1 1 1 0 1 0 1 0 1 1 1 1 1 1 1 1 0 1 1 1 17
Ch_4_Cl_5 - Total 5 3 4 3 1 1 2 1 1 0 5 2 3 3 1 2 4 1 0 5 4 4 54
Ch_5 - Click here 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 3
Ch_5 - Click here 1 0 1 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 1 0 0 4
Ch_5 - Click here 1 0 1 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 4
Ch_5 - Click here 1 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 1 1 5
Ch_5 - Click here 1 1 1 1 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 1 0 0 8
Ch_5 - Click here 1 1 0 0 1 0 0 0 0 0 1 1 1 1 1 1 0 0 0 1 0 0 9
Ch_5 - Click here 1 1 0 0 0 1 1 0 0 1 1 0 0 1 0 0 0 0 0 1 1 1 9
Ch_5 - Click here 1 1 1 1 0 0 1 0 0 0 1 1 1 1 0 0 0 0 0 1 0 0 9
Ch_5 - Click here 1 1 1 1 0 0 0 0 0 0 1 1 1 0 1 1 0 0 0 1 0 0 9
Ch_5 - Click here 1 1 1 1 0 1 1 0 1 0 1 0 1 0 0 1 0 0 0 1 0 0 10
Ch_5 - Click here 1 0 1 0 0 0 1 0 0 0 1 0 0 1 1 1 1 0 0 1 1 1 10
Ch_5 - Click here 1 1 1 1 0 1 0 0 0 1 1 0 1 1 1 1 0 0 0 1 0 0 11
Ch_5 - Click here 1 1 1 1 0 0 0 0 1 0 1 1 0 0 0 1 1 0 0 1 1 1 11
Ch_5 - Click here 1 1 1 1 0 0 1 0 1 0 1 0 1 1 1 1 0 0 0 1 0 0 11
Ch_5 - Click here 1 1 1 1 0 0 0 0 0 0 1 1 1 1 1 0 0 0 0 1 1 1 11
Ch_5 - Click here 1 1 1 1 0 0 0 1 0 0 1 0 1 1 1 1 1 0 0 1 0 0 11
Ch_5 - Click here 1 1 1 1 1 0 1 0 1 0 1 1 1 1 0 0 1 0 0 1 0 0 12
Ch_5 - Click here 1 1 1 1 0 1 0 0 1 0 1 1 1 1 0 1 1 0 0 1 0 0 12
Ch_5 - Click here 1 1 1 1 0 0 0 0 0 1 1 0 1 1 1 1 0 0 0 1 1 1 12
Ch_5 - Click here 1 1 1 1 0 1 1 0 0 0 1 1 1 1 0 1 1 0 0 1 0 0 12
Ch_5 - Click here 1 1 1 1 0 0 0 0 1 1 1 0 1 1 0 0 1 0 0 1 1 1 12
Ch_5 - Click here 1 1 1 1 0 0 0 1 0 0 1 1 1 1 1 1 1 0 0 1 0 0 12
Ch_5 - Click here 1 1 1 1 0 0 0 1 0 0 1 1 1 1 1 1 1 0 0 1 0 0 12
Ch_5 - Click here 1 1 1 1 1 1 1 0 1 0 1 1 1 1 0 0 0 0 0 1 1 1 14
Ch_5 - Click here 1 1 1 1 0 1 0 0 0 0 1 1 1 1 1 1 1 0 0 1 1 1 14
Ch_5 - Click here 1 1 1 1 0 1 1 0 1 0 1 0 1 1 1 1 1 0 0 1 1 1 15
Ch_5 - Click here 1 1 1 1 0 0 1 0 1 0 1 1 1 1 1 1 1 0 0 1 1 1 15
Ch_5 - Click here 1 1 1 1 0 1 0 1 1 0 1 1 1 1 1 1 1 0 0 1 1 1 16
Ch_5 - Click here 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 0 0 1 1 1 18
Ch_5 - Click here 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 0 0 1 1 1 18
Ch_5 - Click here 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 18
Ch_5 - Click here 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 18
Ch_5 - Click here 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 0 1 1 1 19
Ch_5 - Click here 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 19
Ch_5 - Click here 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 20
Ch_5 - Total 35 31 32 28 10 15 17 9 17 11 35 22 28 29 20 24 20 2 0 35 19 19 457
all - Click here 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
all - Click here 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
all - Click here 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
all - Click here 1 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 3
all - Click here 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 3
all - Click here 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 3
all - Click here 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 3
all - Click here 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 3
all - Click here 1 0 1 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 1 0 0 4
all - Click here 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 1 0 0 4
all - Click here 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 1 0 0 4
all - Click here 1 0 1 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 1 0 0 4
all - Click here 1 0 1 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 4
all - Click here 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 5
all - Click here 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 5
all - Click here 1 0 1 0 0 1 1 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 5
all - Click here 1 1 1 1 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 5
all - Click here 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 1 1 0 0 1 0 0 5
all - Click here 1 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 1 1 5
all - Click here 1 1 1 1 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 1 0 0 6
all - Click here 1 1 1 1 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 6
all - Click here 1 1 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 1 1 1 6
all - Click here 1 1 1 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 1 0 0 6
all - Click here 1 0 1 0 0 0 0 0 0 1 1 0 0 0 1 0 1 0 0 1 0 0 6
all - Click here 1 1 1 1 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 6
all - Click here 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 1 1 1 6
all - Click here 1 0 1 0 0 0 1 1 0 0 1 0 0 0 0 0 1 0 0 1 0 0 6
all - Click here 1 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 1 0 0 1 1 1 6
all - Click here 1 0 1 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 1 0 0 6
all - Click here 1 1 1 1 0 0 0 1 0 0 1 0 0 0 0 1 0 0 0 1 0 0 7
all - Click here 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0 1 1 0 0 1 0 0 7
all - Click here 1 1 1 1 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 1 0 0 7
all - Click here 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 1 1 7
all - Click here 1 1 0 0 0 0 0 0 0 0 1 1 1 1 0 0 1 0 0 1 0 0 7
all - Click here 1 0 1 0 0 0 1 0 0 1 1 0 0 0 0 1 1 0 0 1 0 0 7
all - Click here 1 0 1 0 0 1 0 0 0 0 1 0 0 0 0 0 1 0 0 1 1 1 7
all - Click here 1 1 1 1 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 1 1 1 8
all - Click here 1 0 1 0 0 0 0 0 0 0 1 1 0 0 0 1 1 0 0 1 1 1 8
all - Click here 1 1 1 1 0 0 0 1 0 0 1 0 0 1 0 0 1 0 0 1 0 0 8
all - Click here 1 1 1 1 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 1 1 1 8
all - Click here 1 1 1 1 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 1 0 0 8
all - Click here 1 1 0 0 0 1 0 0 0 0 1 1 1 1 1 0 1 0 0 1 0 0 9
all - Click here 1 1 1 1 0 1 0 0 0 0 1 0 1 1 0 0 1 0 0 1 0 0 9
all - Click here 1 1 1 1 0 0 0 0 0 0 1 1 0 0 0 0 1 0 0 1 1 1 9
all - Click here 1 1 1 1 0 0 0 0 0 0 1 1 1 1 0 1 0 0 0 1 0 0 9
all - Click here 1 1 1 1 0 0 0 0 0 0 1 0 1 1 0 1 1 0 0 1 0 0 9
all - Click here 1 1 0 0 1 0 0 0 0 0 1 1 1 1 1 1 0 0 0 1 0 0 9
all - Click here 1 1 0 0 0 1 1 0 0 1 1 0 0 1 0 0 0 0 0 1 1 1 9
all - Click here 1 1 1 1 0 0 1 0 0 0 1 1 1 1 0 0 0 0 0 1 0 0 9
all - Click here 1 1 1 1 0 0 0 0 0 0 1 1 1 0 1 1 0 0 0 1 0 0 9
all - Click here 1 1 1 1 0 0 0 0 0 1 1 0 1 1 0 0 0 0 0 1 1 1 10
all - Click here 1 1 1 1 0 1 0 0 1 0 1 1 1 1 0 0 0 0 0 1 0 0 10
all - Click here 1 1 1 1 0 0 0 0 1 0 1 1 1 0 0 0 0 0 0 1 1 1 10
all - Click here 1 1 1 1 0 1 0 0 0 1 1 0 0 1 0 1 1 0 0 1 0 0 10
all - Click here 1 1 1 1 1 0 0 0 1 0 1 0 0 0 0 0 1 0 0 1 1 1 10
all - Click here 1 1 1 1 0 0 0 0 1 0 1 1 1 0 0 0 0 0 0 1 1 1 10
all - Click here 1 1 1 1 0 0 1 1 0 0 1 0 0 1 0 1 1 0 0 1 0 0 10
all - Click here 1 1 1 1 0 1 0 0 0 0 1 0 1 1 1 0 1 0 0 1 0 0 10
all - Click here 1 1 1 1 0 0 0 0 0 0 1 0 1 1 0 0 1 0 0 1 1 1 10
all - Click here 1 1 1 1 0 0 0 1 0 1 1 0 0 0 0 0 1 0 0 1 1 1 10
all - Click here 1 0 1 0 0 0 0 0 1 1 1 1 1 1 0 1 1 0 0 1 0 0 10
all - Click here 1 1 1 1 0 0 0 0 0 1 1 0 0 0 0 1 1 0 0 1 1 1 10
all - Click here 1 1 1 1 0 1 0 0 0 0 1 1 0 0 0 0 1 0 0 1 1 1 10
all - Click here 1 0 1 0 0 0 1 0 0 0 1 0 0 1 1 1 1 0 0 1 1 1 10
all - Click here 1 1 1 1 0 0 1 0 0 0 1 0 0 0 0 1 1 0 0 1 1 1 10
all - Click here 1 1 1 1 0 1 1 0 1 0 1 0 1 0 0 1 0 0 0 1 0 0 10
all - Click here 1 1 1 1 0 0 0 0 0 0 1 0 1 1 0 1 1 0 0 1 1 1 11
all - Click here 1 1 1 1 0 0 0 0 1 0 1 0 0 1 1 0 1 0 0 1 1 1 11
all - Click here 1 1 1 1 0 0 1 0 0 0 1 1 0 1 0 1 0 0 0 1 1 1 11
all - Click here 1 1 1 1 0 0 0 0 0 0 1 0 1 1 0 1 1 0 0 1 1 1 11
all - Click here 1 1 1 1 0 1 0 0 0 1 1 0 0 0 0 1 1 0 0 1 1 1 11
all - Click here 1 1 1 1 0 0 0 0 0 0 1 0 1 1 0 1 1 0 0 1 1 1 11
all - Click here 1 1 1 1 0 1 0 0 0 0 1 0 1 1 0 0 1 0 0 1 1 1 11
all - Click here 1 1 1 1 0 1 0 0 0 1 1 0 1 1 1 1 0 0 0 1 0 0 11
all - Click here 1 1 1 1 0 0 0 0 1 0 1 1 0 0 0 1 1 0 0 1 1 1 11
all - Click here 1 1 1 1 0 0 1 0 1 0 1 0 1 1 1 1 0 0 0 1 0 0 11
all - Click here 1 1 1 1 0 0 0 0 0 0 1 1 1 1 1 0 0 0 0 1 1 1 11
all - Click here 1 1 1 1 0 0 0 1 0 0 1 0 1 1 1 1 1 0 0 1 0 0 11
all - Click here 1 1 1 1 0 1 1 0 0 0 1 1 1 1 0 1 1 0 0 1 0 0 12
all - Click here 1 1 1 1 0 0 0 0 0 1 1 0 1 1 1 1 0 0 0 1 1 1 12
all - Click here 1 1 1 1 1 0 1 0 1 0 1 1 1 1 0 0 1 0 0 1 0 0 12
all - Click here 1 1 1 1 0 1 0 0 1 0 1 1 1 1 0 1 1 0 0 1 0 0 12
all - Click here 1 1 1 1 0 0 0 0 1 1 1 0 1 1 0 0 1 0 0 1 1 1 12
all - Click here 1 1 1 1 0 0 0 1 0 0 1 1 1 1 1 1 1 0 0 1 0 0 12
all - Click here 1 1 1 1 0 0 0 1 0 0 1 1 1 1 1 1 1 0 0 1 0 0 12
all - Click here 1 1 1 1 0 1 0 0 1 0 1 1 1 1 1 1 1 0 0 1 0 0 13
all - Click here 1 1 1 1 0 1 0 0 0 1 1 1 1 1 1 1 1 0 0 1 0 0 13
all - Click here 1 1 1 1 0 1 0 0 1 0 1 1 0 1 0 1 1 0 0 1 1 1 13
all - Click here 1 1 1 1 0 0 0 1 0 1 1 1 1 1 0 1 1 0 0 1 1 1 14
all - Click here 1 1 1 1 0 1 0 0 0 0 1 1 1 1 1 1 1 0 0 1 1 1 14
all - Click here 1 1 1 1 1 1 1 0 1 0 1 1 1 1 0 0 0 0 0 1 1 1 14
all - Click here 1 1 1 1 0 1 0 1 1 0 1 1 1 1 0 1 1 0 0 1 1 1 15
all - Click here 1 1 1 1 0 1 0 1 1 0 1 1 1 1 0 1 1 0 0 1 1 1 15
all - Click here 1 1 1 1 0 1 1 0 1 0 1 1 1 1 0 1 1 0 0 1 1 1 15
all - Click here 1 1 1 1 0 1 1 0 1 0 1 0 1 1 1 1 1 0 0 1 1 1 15
all - Click here 1 1 1 1 0 0 1 0 1 0 1 1 1 1 1 1 1 0 0 1 1 1 15
all - Click here 1 1 1 1 1 1 1 0 1 0 1 1 1 1 0 1 1 0 0 1 1 1 16
all - Click here 1 1 1 1 1 1 1 0 1 0 1 0 1 1 0 1 1 1 0 1 1 1 16
all - Click here 1 1 1 1 1 0 0 1 1 1 1 1 1 1 0 1 1 0 0 1 1 1 16
all - Click here 1 1 1 1 0 1 1 1 0 0 1 1 1 1 1 1 1 0 0 1 1 1 16
all - Click here 1 1 1 1 0 0 1 0 1 1 1 1 1 1 1 1 1 0 0 1 1 1 16
all - Click here 1 1 1 1 0 1 1 0 1 0 1 1 1 1 1 1 1 0 0 1 1 1 16
all - Click here 1 1 1 1 0 0 1 0 1 1 1 1 1 1 1 1 1 0 0 1 1 1 16
all - Click here 1 1 1 1 0 1 0 1 1 0 1 1 1 1 1 1 1 0 0 1 1 1 16
all - Click here 1 1 1 1 0 0 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 17
all - Click here 1 1 1 1 0 1 1 1 1 0 1 1 1 1 1 1 1 0 0 1 1 1 17
all - Click here 1 1 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 0 0 1 1 1 17
all - Click here 1 1 1 1 1 1 1 1 1 1 1 0 1 1 0 1 1 0 0 1 1 1 17
all - Click here 1 1 1 1 1 0 1 0 1 1 1 1 1 1 1 1 1 0 0 1 1 1 17
all - Click here 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 17
all - Click here 1 1 1 1 1 1 1 0 1 0 1 1 1 1 1 1 1 0 0 1 1 1 17
all - Click here 1 1 1 1 0 1 1 0 1 1 1 1 1 1 1 1 1 0 0 1 1 1 17
all - Click here 1 1 1 1 1 0 1 0 1 1 1 1 1 1 1 1 1 0 0 1 1 1 17
all - Click here 1 1 1 1 1 0 1 0 1 0 1 1 1 1 1 1 1 1 0 1 1 1 17
all - Click here 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 18
all - Click here 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 0 0 1 1 1 18
all - Click here 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 0 0 1 1 1 18
all - Click here 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 18
all - Click here 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 0 0 1 1 1 18
all - Click here 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 18
all - Click here 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 18
all - Click here 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 0 0 1 1 1 18
all - Click here 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 0 0 1 1 1 18
all - Click here 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 18
all - Click here 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 18
all - Click here 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 19
all - Click here 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 19
all - Click here 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 19
all - Click here 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 19
all - Click here 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 19
all - Click here 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 0 1 1 1 19
all - Click here 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 19
all - Click here 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 20
all - Click here 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 20
all - Total 134 109 125 103 32 54 54 35 56 43 127 71 84 89 52 80 92 6 0 127 79 79 1630

TCA Corpus properties

TODO Countries in TCA Corpus

Show the code
map <- countries_tca_corpus |>
    mutate(
        log_count = log(count)
    ) |>
    map_country_codes(
        values = "log_count",
        map_type = "countries"
    )
Warning: Some values were not matched unambiguously: XK
Warning in map_country_codes(mutate(countries_tca_corpus, log_count = log(count)), : The following countries are not in the world dataset: 
AIA, BLM, BMU, CXR, GGY, GIB, IOT, MCO, MHL, MDV, NFK, NRU, SMR, TKL, TUV, UMI, VAT, VGB, WLF, NA
and will therefore not be plotted!
Show the code
map

Topics in corpus

Show the code
#|

cs <- cumsum(prim_topics_tca_corpus$count)
cs |>
    plot(
        type = "l",
        xlab = "Topic",
        ylab = "Cumulative Count",
        main = "Cumulative Topics in TCA Corpus"
    )

abline(
    h = 0.95 * cs[length(cs)],
    v = min(which(cs > 0.95 * cs[length(cs)])),
    col = "red"
)

text(
    x = 0.5 * length(cs),
    y = 0.95 * cs[length(cs)],
    pos = 3,
    labels = "95% of the corpus",
    col = "red"
)

Show the code
#|

prim_topics_tca_corpus |>
    relocate(count, .after = "topic_id") |>
    DT::datatable(
        extensions = c(
            "Buttons",
            "FixedColumns",
            "Scroller"
        ),
        options = list(
            dom = "Bfrtip",
            buttons = list(
                list(
                    extend = "csv",
                    filename = fn
                ),
                list(
                    extend = "excel",
                    filename = fn
                ),
                list(
                    extend = "pdf",
                    filename = fn,
                    orientation = "landscape",
                    customize = DT::JS(
                        "function(doc) {",
                        "  doc.defaultStyle.fontSize = 5;", # Change the font size
                        "}"
                    )
                ),
                "print"
            ),
            scroller = TRUE,
            scrollY = JS("window.innerHeight * 0.7 + 'px'"),
            scrollX = TRUE,
            fixedColumns = list(leftColumns = 4)
        ),
        escape = FALSE
    )
Warning in instance$preRenderHook(instance): It seems your data is too big for
client-side DataTables. You may consider server-side processing:
https://rstudio.github.io/DT/server.html

SubFields in Corpus

Show the code
#|

prim_topics_tca_corpus |>
    mutate(
        topic_id = NULL,
        topic_name = NULL,
        keywords = NULL,
        summary = NULL,
        wikipedia_url = NULL
    ) |>
    group_by(
        subfield_id,
        subfield_name,
        field_id,
        field_name,
        domain_id,
        domain_name
    ) |>
    summarise(
        count = sum(count)
    ) |>
    arrange(desc(count)) |>
    relocate(count, .after = "subfield_id") |>
    DT::datatable(
        extensions = c(
            "Buttons",
            "FixedColumns",
            "Scroller"
        ),
        options = list(
            dom = "Bfrtip",
            buttons = list(
                list(
                    extend = "csv",
                    filename = fn
                ),
                list(
                    extend = "excel",
                    filename = fn
                ),
                list(
                    extend = "pdf",
                    filename = fn,
                    orientation = "landscape",
                    customize = DT::JS(
                        "function(doc) {",
                        "  doc.defaultStyle.fontSize = 5;", # Change the font size
                        "}"
                    )
                ),
                "print"
            ),
            scroller = TRUE,
            scrollY = JS("window.innerHeight * 0.7 + 'px'"),
            scrollX = TRUE,
            fixedColumns = list(leftColumns = 4)
        ),
        escape = FALSE
    )
  • TODO: Histogram / table with topics identified to identify non-relevant papers

Reuse

Citation

BibTeX citation:
@report{m krug2024,
  author = {M Krug, Rainer},
  title = {Data {Management} {Report} {Transformative} {Change}
    {Assessment} {Corpus} - {SOD}},
  date = {2024-02-19},
  doi = {10.5281/zenodo.10251349},
  langid = {en},
  abstract = {The literature search for the assessment corpus was
    conducted using search terms provided by the experts and refined in
    co-operation with the Knowldge and Data task force. The search was
    conducted using {[}OpenAlex{]}(https://openalex.org), scripted from
    {[}R{]}(https://cran.r-project.org) to use the
    {[}API{]}(https://docs.openalex.org). Search terms for the following
    searches were defined: **Transformative Change**, **Nature /
    Environment** and **additional search terms for individual chapters
    and sub-chapters** To assess the quality of the corpus, sets of
    key-papers were selected by the experts to verify if these are in
    the corpus. These key-papers were selected per chapter / sub-chapter
    to ensure that the corpus is representative of each chapter.}
}
For attribution, please cite this work as:
M Krug, Rainer. 2024. “Data Management Report Transformative Change Assessment Corpus - SOD.” Report Transformative Change Assessment Corpus. https://doi.org/10.5281/zenodo.10251349.